Sensor based clear path robot guide

ABSTRACT

A guide system is provided that uses a plurality of sensors to identify and determine a clear path for an ambulatory vision impaired person. The system includes one or more wheels that rotate to propel the system, a platform supported by the one or more wheels and housing a processor, a rigid harness with a haptic feedback grip that is positioned to be grasped by an operator, and one or more sensors configured to sense information about the environment. In operation, the processor analyzes information sensed by the sensors to identify object in the path of the guide system and sends messages to the operator to allow the operator to avoid the identified objects. The messages may be sent to the operator via the haptic feedback grip or audibly via a speaker or via a wireless connection to a haptic or audio device being worn by the operator.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage entry of International ApplicationNo. PCT/US17/056640, filed on Oct. 13, 2017, which claims priority toU.S. Provisional Patent App. No. 62/408,609, filed on Oct. 14, 2016—theentirety of which is hereby incorporated herein by reference.

BACKGROUND Field of the Invention

The present invention generally relates to robotics and moreparticularly relates to a sensor based clear path robot guide for thevision impaired.

Related Art

Conventional assistive technology available to ambulatory visionimpaired people has been around for decades and centuries. Specifically,the white cane device has been used by ambulatory vision impaired forover two centuries while trained guide dogs have been used by the visionimpaired for seven or more decades. These assistive technologies arehelpful, but they have not been updated or improved since theirrespective introductions. A white cane investigates the environment bysweeping side to side in a three foot width pattern. The white cane usersenses objects in his or her path by coming in contact with the whitecane. The guide dog guides a person in the direction that the personcommands the dog to take. The guide dog avoids objects in the givenpath. With the dog guide method, the vision impaired person does notmake physical contact with the environment.

The World Health Organization (WHO) reported in 2014 that approximately285 million people (about 4% of the total population) worldwide wereestimated to be visually impaired, with 39 million blind and 246 millionsuffering from low vision. The development of technologies that provideimproved tactile, orientation and sensory perception to enable safenavigation is important for individuals who are vision impaired.However, most of the commercially available products to assist thevision impaired in object avoidance and safe navigation are similar tothat used more than fifty years ago.

There are three main product types that aid vision impaired individualsfor safe navigation through their surroundings. These include electronictravel aids (ETAs), electronic orientation aids (EOAs), and positionlocator devices (PLDs). Many researchers and ETA product developersbelieve these traditional methods should be replaced by productscomprised of a variety of sensors, cameras and other technologiescapable of gathering and relaying information about proximalsurroundings for clear path navigation. Vision impaired individuals havehad access to commercially available ETA's such as ultrasound sensors,other sonic-related guides, infrared sensors and a variety ofcamera/vision technologies, but wide-scale adoption of any ETA has notoccurred to significantly replace traditional methods. 8 It is not fullyunderstood if the lack of significant adoption is due to a lack ofconfidence in ETAs or their cost.

The emergence of artificial intelligence and improvement,miniaturization and reduced cost of laser-based sensors has given morehope to researchers and ETA product developers that devices withimproved capabilities for object avoidance and clear path navigation caneventually be made to replace traditional methods. Additionally, thesesame factors, specifically the miniaturization of sensors, have givenrise to wearable navigation systems that some assume will be moreadoptable than previous ETA's. Therefore, what is needed is a modernassistive technology for use by ambulatory vision impaired people.

SUMMARY

In order to solve the problems discussed above, the present disclosureis directed toward a robotic device that is capable of providing clearpath detection and object avoidance as an alternative to a guide dog,white cane and other ETA's for vision impaired individuals. The roboticdevice is comprised of one or more of: optical, proximal and ultrasonicsensors, GPS, WiFi, an accelerometer, a central processing unit and awheelbase capable of tactile feedback. The collection of integratedsensors and communication devices are able to perceive and reveal richdetails about a vision impaired person's surroundings, a significantimprovement over a conventional guide dog or white cane.

The guide system described herein provides more detail about theoperator's environment. It can provide clear path detection, objectavoidance, and computerized information as it navigates a clear safepath. The guide system includes a rigid but collapsible harness for theoperator to grasp and thereby engage a direct tactile connection to thesurface. The rigid harness is attached to a base with one or morewheels. The base comprises a housing that completely or partiallysurrounds and protects a motor and a processor that analyzes informationfrom the one or more sensors such as Lidar Radar Sensors, ultrasonicsensors, accelerometers, and Proximity Sensors. The processor may alsoanalyze information from one or more wireless communication devicesconnected to the guide system via WIFI or Bluetooth or the like. Forexample, the operator may wear a Bluetooth earpiece or wristwatch toreceive auditory feedback from the guide system. The combination of theone or more wheels upon the surface and the rigid harness connected tothe one or more wheels provides tactile feedback from the surfacedirectly to the operator when the operator grasps the harness.

In operation, the operator grasps the rigid harness and begins to pushthe guide system that rolls along the surface using its wheels, whichprovides a direct tactile connection between the operator and thesurface. As the operator proceeds forward along a primary course oftravel, the sensors attached to the guide system provide sensor data toa processor that analyzes the sensor data to identify a clear path forthe operator to travel. Feedback to the operator may be auditory (beeps,sounds, speech) to alert the operator to the presence of an object inthe primary course of travel. Feedback to the operator may also bethough a haptic user interface in the harness or other device.

The guide system provides a safe and clear path for the visuallyimpaired and provides independence and confidence. The guide system canguide a visually impaired person in a path from point A to point B,avoid obstacles on the surface, as well as warn of obstacles on eitherside of and above the operator. The guide system is configured to detectdrop-offs such as curbs and stairs and find openings such as doors andgates, and identify the presence of objects. The harness, base andwheels of the guide system can be used as a navigation device. With asingle sensor and user interface feedback system (e.g., a speaker), theguide system provides basic object detection and avoidance.

Advantageously, to solve the problems of the conventional assistivetechnology, described herein are a system, apparatus and methods of usefor a robot guide that uses a plurality of sensors to identify anddetermine a clear path for an ambulatory vision impaired person. In oneaspect, the robot guide includes a set of wheels that rotate to propelthe robot, a platform supported by the set of wheels and housingcontrolling electronics for the robot, a handle that is positioned to begrasped by the operator and at least two sensors that are configured tosense information about the guide system environment, for example,information about objects in a forward path of the guide system. Theguide system also includes a processor configured to analyze informationfrom the sensors and construct and send messages to the operator via oneor more user interfaces. For example, the processor may send an audiblemessage to the operator via a speaker or a tactile message to theoperator via the handle or via an interface device being worn by theoperator (e.g., a watch or headset or earpiece, etc.).

Other features and advantages of the present invention will become morereadily apparent to those of ordinary skill in the art after reviewingthe following detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The structure and operation of the present invention will be understoodfrom a review of the following detailed description and the accompanyingdrawings in which like reference numerals refer to like parts and inwhich:

FIG. 1A is a block diagram illustrating an example sensor based clearpath robot guide system according to an embodiment of the invention;

FIG. 1B is a block diagram illustrating an example sensor based clearpath robot guide system with a plurality of sensor zones according to anembodiment of the invention;

FIG. 1C is a block diagram illustrating an example sensor based clearpath robot guide system with a forward sensor angle width according toan embodiment of the invention;

FIG. 1D is a block diagram illustrating an example sensor based clearpath robot guide system with a forward sensor angle height according toan embodiment of the invention;

FIG. 2 is a block diagram illustrating an alternative example sensorbased clear path robot guide system according to an embodiment of theinvention;

FIG. 3A is a block diagram illustrating an alternative example sensorbased clear path robot guide system according to an embodiment of theinvention;

FIG. 3B is a block diagram illustrating an alternative example sensorbased clear path robot guide system according to an embodiment of theinvention; and

FIG. 4 is a block diagram illustrating an example wired or wirelessprocessor enabled device that may be used in connection with variousembodiments described herein.

The following element numbers are used throughout the drawings:

-   -   10—guide system    -   15—wheel    -   20—platform    -   25—housing    -   30—harness    -   35—grip    -   37—accelerator/decelerator    -   40—sensors    -   45—CPU    -   50—motor    -   55—tray    -   60—tilt preventer    -   65—user interface    -   70—wireless communication device

DETAILED DESCRIPTION

Certain embodiments disclosed herein provide for a sensor based clearpath robot guide system that is used as an ambulatory assistance aid forthe vision impaired. After reading this description it will becomeapparent to one skilled in the art how to implement the invention invarious alternative embodiments and alternative applications. However,although various embodiments of the present invention will be describedherein, it is understood that these embodiments are presented by way ofexample only, and not limitation. As such, this detailed description ofvarious alternative embodiments should not be construed to limit thescope or breadth of the present invention as set forth in the appendedclaims.

FIG. 1A is a block diagram illustrating an example sensor based clearpath robot guide system 10 according to an embodiment of the invention.In the illustrated embodiment, the system 10 comprises two wheels 15connected by an axle 22 that supports a platform 20 that in turnsupports a housing 25. The axle 22, platform 20 and housing 25 arepositioned above the ground and are connected to a harness 30 thatextends upward away from the platform 20 and housing 25 and the axle 22.At a distal end (away from the housing) the harness 30 comprises a grip35 that is configured for an operator to grasp. In one embodiment, thegrip 35 may have an integrated accelerator/decelerator 37 configured todrive the motor 50. The guide system 10 also comprises plural sensors 40that are connected to the platform 20, housing 24 and/or harness 30 atdifferent locations. In one embodiment, the plural sensors 40 mayinclude one or more of LIDAR radar sensors, ultrasonic sensors,accelerometers, proximity sensors, radar sensors, infrared sensors,imaging sensors, GPS sensors, and the like. Preferably, a least one ofthe plurality of sensors 40 is positioned at different elevations alongthe guide system 10 to maximize the combined coverage of the pluralsensors 40. In one embodiment, the harness 30 may also support one ormore trays 55 or baskets or other helpful containers for storage ofitems by the operator.

The housing 25 may also support plural sensors 40 to expand the combinedcoverage of the total sensors 40 that are connected to the guide system10. The housing 25 also defines an interior cavity in which electronicdevices are located. Electronic devices may include devices such as acentral processing unit (“CPU”) 45, one or more additional sensors 40such as accelerometers, and global position system (“GPS”) receivers,and communication devices such as blue tooth radios and the like. Theinterior cavity may also include the motor 50 and a power source (e.g.,a battery) configured to power the motor 50 and other electronics on theguide system 10.

The various sensors 40 that are connected to the guide system 10 caninclude ultrasonic sensors, radar sensors, camera sensors, infraredsensors, light detection and ranging (“LIDAR”) sensors and the like.These sensors 40 can be combined in any variety to maximize the sensorinput to the CPU 45 in order to carry out the function of the guidesystem 10.

In operation, the guide system 10 senses and analyzes environmentalinformation and provides feedback to the operator of the guide system10. Haptic or auditory feedback may be provided to the operator toindicate how to navigate through the environment. In one embodiment, theguide system 10 provides feedback to the operator to facilitate theoperator following a predetermined route. This can be accomplished bythe processor 45 identifying a predetermined route and receiving andanalyzing GPS information to provide feedback to the operator regardingmacro adjustments to the current route (e.g., turn right, turn left).The processor 45 also analyzes sensor data to provide feedback to theoperator regarding micro adjustments to the current route (e.g., stepdown, step up, veer right, veer left, stop, etc.). Micro adjustments canadvantageously allow the operator to avoid obstacles while maintainingcourse.

In one embodiment, one or more motors 50 may be positioned within thehousing 25 to allow the guide system 10 to turn the wheels 15 and propelthe guide system 10 and thereby pull the operator along a predeterminedroute. Advantageously, the processor 45 in the housing is configured tocontrol the one or more motors 50 to turn the wheels 15 synchronouslyand/or asynchronously/individually. For example, the processor 45 maycontrol the wheels 15 to turn in opposite directions to rapidly turn theguide system 10 or the processor 45 may control the wheels 15 to turn inthe same direction at different rates to slowly turn the guide system 10in a desired arc.

FIG. 1B is a block diagram illustrating an example sensor based clearpath robot guide system 10 with a plurality of sensor zones according toan embodiment of the invention. In the illustrated embodiment, the guidesystem 10 has a plurality of sensors 40. There may be one or moresensors 40 per sensor zone. The various sensor zones may have gapsbetween them, may be adjacent, or may overlap by a small or largeportion. In one embodiment where the sensor zones are adjacent, thelowest zone 1 extends from the surface upon which the wheels 15 rest(i.e., 0 inches) to 24 inches, the next zone 2 is from 24 inches to 48inches and the next zone 3 is from 48 inches to 72 inches. There may beadditional or fewer zones in alternative embodiments. In one embodiment,the one or more sensors 40 in each zone may collectively providefeedback to the operator by a specific means. For example, zone 1sensors 40 may provide auditory feedback to the operator via a Bluetoothearpiece while the zone 2 sensors 40 may provide haptic feedback to theoperator via the grip 35 while the zone 3 sensors 40 may provideauditory feedback via one or more speakers.

FIG. 1C is a block diagram illustrating an example sensor based clearpath robot guide system 10 with a forward sensor angle width accordingto an embodiment of the invention. In the illustrated embodiment, theforward sensor has an angle of 8°. Advantageously, the angle θ may varyin alternative embodiments, for example the angle θ may vary from 10° to180°. In the illustrated embodiment, one or more sensors 40 areconfigured to sense information within the angle θ and provide sensoroutput to the processor 45.

FIG. 1D is a block diagram illustrating an example sensor based clearpath robot guide system 10 with a forward sensor angle height accordingto an embodiment of the invention. In the illustrated embodiment, theforward sensor has an angle of ϕ°. Advantageously, the angle ϕ may varyin alternative embodiments, for example the angle θ may vary from 10° to180°. In the illustrated embodiment, one or more sensors 40 areconfigured to sense information within the angle ϕ and provide sensoroutput to the processor 45.

FIG. 2 is a block diagram illustrating an alternative example sensorbased clear path robot guide system 10 according to an embodiment of theinvention. Elements previously described will not be further describedwith respect to FIG. 2 . In the illustrated embodiment, the guide system10 comprises a telescoping harness 30 that adjusts to the height of theoperator. Additionally, the guide system 10 comprises a tilt preventer60 that extends from a back side of the platform 20 or housing 25. Thetilt preventer 60 advantageously performs multiple functions. A firstfunction is that the tilt preventer 60 functions as a kick stand toallow the guide system 10 to independently stand upright withoutintervention by an operator. A second function is that the tiltpreventer functions 60 as a brake when the operator tilts the guidesystem 10 backward to drag the tilt preventer 60 against the surfaceupon which the guide system 10 is moving and thereby increase friction.A third function is that the tilt preventer 60 functions as a mechanicalassist when lowering the guide system 10 down a step or curb and alsofunctions as a mechanical assist when lifting the guide system 10 up astep or curb.

FIG. 3A is a block diagram illustrating an alternative example sensorbased clear path robot guide system 10 according to an embodiment of theinvention. Elements previously described will not be further describedwith respect to FIG. 3A. In the illustrated embodiment, the guide system10 comprises a user interface 65. The user interface 65 may includephysical buttons, a viewing screen, a touch screen, colored lights(e.g., light emitting diodes (“LEDs”)), vibration surfaces and the like.In the illustrated embodiment, the user interface 65 is positioned onthe housing 25. In an alternative embodiment, the user interface 65 maybe connected to the harness 30, for example near the distal end andintegrated with the grip 35. Additionally, portions of the userinterface 65 may be separated, for example with a haptic interfaceportion integrated with the grip 35 and an illuminated portionpositioned on the housing 25 and an auditory portion (e.g., a speaker)integrated near the grip 35 and/or near the housing 25 or along theharness 30. Alternative combinations are also possible.

FIG. 3B is a block diagram illustrating an alternative example sensorbased clear path robot guide system 10 according to an embodiment of theinvention. In the illustrated embodiment, the guide system 10 comprisesa single wheel 15 with an axle 22. A harness 30 is connected to the axle22 at a proximal end of the harness 30 and the wheel 15 and axle 22support the harness 30 that extends upward toward a grip 35 that ispositioned at a distal end of the harness 30. The guide system 10 mayalso include a tilt preventer 60.

Additionally, one or more sensors 40 are connected to or integrated withthe guide system 10, for example positioned in different sensor zones.The one or more sensors 40 are communicatively coupled with a processor45 that is configured to receive sensor information from the one or moresensors 40 and analyze the sensor information to provide feedback to anoperator of the guide system 10. The processor 45 is configured toprovide feedback to an operator of the guide system 10 via one or moreuser interfaces 65, which may include a haptic feedback user interface65 portion and an audio feedback user interface 65 portion. Variousportions of the user interface 65 may be positioned anywhere along theguide system 10, for example in or on the grip 35 and in or on theharness 30.

In one embodiment, a motor 50 may be optionally included in the guidesystem 10. In such an embodiment, the motor is configured to drive thewheel 15 to facilitate acceleration by the operator of the guide system10 or to provide resistance to the operator of the guide system 10 andthereby facilitate deceleration by the operator of the guide system 10.Advantageously, the processor 45 is configured to control the motor toprovide variable speed acceleration or deceleration as well as completestop braking.

In one embodiment, the processor 45 of the guide system 10 iscommunicatively coupled to an external wireless communication device 70and is configured to communicate with the wireless communication device70 to receive operating instructions. For example, the wirelesscommunication device 70 may include a GPS capability and a mapsapplication that creates a route for the operator of the guide system10. In such an embodiment, the wireless communication device 70 sendsinstructions to the processor 45 of the guide system 10 and theprocessor 45 of the guide system 10 carries out those instructions, forexample to provide route feedback to the operator of the guide system10. Advantageously, in combination with the wireless communicationdevice 70 managing the route, the guide system 10 uses its processor 45to monitor sensor information from the various sensors 40 and provideclear path feedback to the operator of the guide system 10.

In one embodiment, the processor 45 of the guide system 10 is configuredto monitor and receive sensor information from the various sensors 40and provide the sensor information to the wireless communication device70 for analysis. In such an embodiment, the wireless communicationdevice 70 comprises the user interface 65 and the wireless communicationdevice 70 is configured to provide audio and/or haptic feedback to theoperator of the guide system 10.

Example Embodiments

In one embodiment, the guide system 10 operates on an expandableplatform that allows additional sensors and user interfaces to beincorporated as needed. The guide system 10 provides a physicalconnection between the operator, who grasps the harness, and the surfaceupon which the operator is walking. The guide system 10 uses acombination of sensors to identify objects in the path of the operatorand uses the user interface systems to provide instructions to theoperator to facilitate object avoidance. Additionally, the guide system10 is configured to follow a predetermined route to facilitate point topoint travel by the operator.

The guide system 10 comprises a rigid harness that is configured toextend out and adjust to the characteristics of the operator, forexample the height of the operator. The rigid harness is attached to thehousing, which is supported on one or more wheels 15. Advantageously,vibration from the surface travels through the wheels 15 and the rigidharness to provide a direct tactile connection between the operator andthe surface.

In one embodiment, attached to the top of the housing is a LIDAR Radarwhich is the primary sensor of the device. In addition to the LIDARRadar, the device also uses at least one ultrasonic sensor positioned ontop of the LIDAR Radar. The ultrasonic sensor on top of the LIDAR Radaradvantageously senses information about any objects in the path oftravel and the processor analyzes the sensed information to identify aprimary clear path for travel by way of detecting any objects in thecurrent path and determining any necessary micro course adjustments toensure object avoidance.

In one embodiment, a wired or wireless ear piece with a speaker (e.g.,Bluetooth) is worn by the operation and audio feedback is provided fromthe guide system 10 to the operator via the ear piece. In oneembodiment, the ultrasonic sensor may provide audio feedback directly tothe earpiece. In an alternative embodiment, the processor generates anaudio message based on an analysis of the sensor data and provides theaudio message to the operator via a connection (wired or wireless) tothe earpiece speaker. The content of an audio message can be a simplesound or beep or it may be synthesized speech.

In one embodiment, a separate wireless communication device (e.g., asmart phone) provides GPS location information, map guidance, andauditory location and/or directions/instructions in connection with theguide system 10. In such an embodiment, the separate wirelesscommunication device provides the operator with the point-to-pointtravel capability and macro course adjustments while the guide system 10provides the micro course adjustments for object avoidance and directenvironment navigation.

In an alternative embodiment, LIDAR Radar performs both the micro courseadjustments and the macro course adjustments in connection with apredetermined route (e.g., a preplanned Google map travel route). TheLIDAR Radar accordingly provides sensor data for analysis to provide aclear path for travel and object avoidance.

Importantly, the combination of sensors on the guide system 10 areconfigured to sense information about the immediate environment toenable surface navigation. Because all objects are in some way attachedto the surface due to gravity, objects such as walls, tables, chairs,trees, signs, etc. are all positive masses that are connected to thesurface and capable of being sensed by the combination of sensors on theguide system 10. Similarly, open doorways, open manholes, side streets,alleyways, etc. are negative masses that are also capable of beingsensed by the combination of sensors on the guide system 10.Accordingly, the combination of sensors on the guide system 10 scan forpositive and negative masses appearing in the primary course of travel.The combination of sensors on the guide system 10 also scan forincreases or decreases in elevation (surface).

In one embodiment, the combination of sensors on the guide system 10sense information in the forward direction. For example, the sensors maybe configured to sense information in an angular direction that is fivedegrees to the left and right of the center of the primary course oftravel of the guide system 10. The sensors may be configured to senseinformation in an angular direction that is five degrees upward anddownward of the center of the primary course of travel of the guidesystem 10. The sensors may also be configured to sense information atwider or narrower angles to the left, right, upward and downward.

Advantageously, forward scanning detects objects that are within theprimary course of travel of the guide system 10. Additionally, a widerangle on the upward scan can assist the operator by detecting lowhanging branches and other objects extending into the primary course oftravel from a sideward or topward angle. The processor is advantageouslyconfigured to analyze the sensor data and determine an X,Y footprint ofan object and plan appropriate micro course adjustments to allow theoperator to take evasive action. In one embodiment, information aboutobjects in the path of an operator is stored in memory in connectionwith location data for the object so that the presence of the object canbe predicted during future travel by the operator or by anotheroperator. Accordingly, information about a first object at a firstlocation may be communicated from a first guide system 10 to a centralserver for storage and subsequently provided to a second guide system 10having a predetermined route that includes the first location.

In one embodiment, the guide system 10 performs its operation byorientating to its initial position and heading, for example bydetermining its GPS coordinates and heading and by also identifying apredetermined course. Subsequently, the guide system 10 periodicallyupdates its position and heading information. The frequency of suchupdates can be configured as needed.

During navigation, the guide system 10 continuously or periodicallyscans its environment (e.g., forward, backward, upward, downward) andthe processor analyzes the sensor data to identify objects in theprimary course of travel and to calculate micro adjustments to theprimary course of travel to avoid such identified objects.Advantageously, because the vast majority of objects in the primarycourse of travel are attached to the surface of the primary course oftravel, the majority of the plural sensors on the guide system 10 can beredundantly trained on the forward surface.

Additionally, during navigation of a repeat trip, information from priortrips of the same course can be employed to predict the location ofobjects in the primary course of travel. Additional information obtainedduring repeat trips can be added to a rich collection of informationabout an overall trip and/or portions of a trip that also appear inother trips such that an object database can be continuously updated andoptimized with information about objects appearing in the primary courseof travel as sensed by one or more guide systems 10 over time.Additionally, during navigation of a first time trip, the guide system10 may be configurable to train additional sensor resources on theforward surface to maximize object detection and avoidance capabilitiesof the guide system 10.

In one embodiment, one or more sensors are trained to scan objectsattached to the forward surface because this region of the primarycourse of travel is the most significant region for forward navigation.The surface refers to the floor, asphalt, concrete, ground, grass, orotherwise of the traveler's environment and primary course of travel.Periodic and/or continuous scanning of the surface and analysis of thesensed information from the surface to identify the presence of one ormore objects is very important because 98% of all objects in the primarycourse of travel are attached to the surface.

Accordingly, the guide system 10 scans the forward surface and theprocessor analyzes the information from the sensors trained on theforward surface and the processor subsequently plans a clear path infront of the operator. The clear path comprises zero or more microadjustments to the current primary course of travel (e.g., the currentheading). The processor of the guide system 10 accordingly constructsand updates an environmental map based on information received from thevarious sensors connected to the guide system 10. Using theenvironmental map, the processor plans a clear path free of obstaclesfor the operator to follow and directions are communicated to theoperator by way of a speaker in an earpiece or haptic feedback to thegrip on the harness or other audio, haptic or even visual cues. In oneembodiment, the rigid harness allows the operator to push the guidesystem 10 at a desired pace while following the micro course adjustmentinstruction received from the processor of the guide system 10 and basedon sensor information about obstacles in the primary course of travel.

In one embodiment, the guide system 10 comprises a kit that can beattached to a wheeled apparatus. The wheeled apparatus may be a walker,running cane, a bicycle or tricycle, an exercise device, and electricbicycle or tricycle, an Elliptigo®, a Segue®, or other such devices. Thekit includes one or more sensors that are configured to senseenvironmental information and provide the sensed information to awireless communication device 70 for analysis. The wirelesscommunication device 70 is configured to receive and analyze the sensorinformation and provide feedback (e.g., haptic and/or audio) and/orinstructions to the operator of the guide system 10. For example, thewireless communication device 70 may provide route guidance (e.g., turnright) or object avoidance (e.g., veer right to avoid a fire hydrant).In one embodiment, route guidance information may originate from amapping application on the wireless communication device while objectavoidance information may originate from the one or more sensorsattached to the wheeled apparatus.

FIG. 4 is a block diagram illustrating an example wired or wirelesssystem 550 that may be used in connection with various embodimentsdescribed herein. For example the system 550 may be used as or inconjunction with a robot guide as previously described with respect toFIGS. 1-3 . The system 550 can be a conventional personal computer,computer server, personal digital assistant, smart phone, tabletcomputer, or any other processor enabled device that is capable of wiredor wireless data communication. Other computer systems and/orarchitectures may be also used, as will be clear to those skilled in theart.

The system 550 preferably includes one or more processors, such asprocessor 560. Additional processors may be provided, such as anauxiliary processor to manage input/output, an auxiliary processor toperform floating point mathematical operations, a special-purposemicroprocessor having an architecture suitable for fast execution ofsignal processing algorithms (e.g., digital signal processor), a slaveprocessor subordinate to the main processing system (e.g., back-endprocessor), an additional microprocessor or controller for dual ormultiple processor systems, or a coprocessor. Such auxiliary processorsmay be discrete processors or may be integrated with the processor 560.

The processor 560 is preferably connected to a communication bus 555.The communication bus 555 may include a data channel for facilitatinginformation transfer between storage and other peripheral components ofthe system 550. The communication bus 555 further may provide a set ofsignals used for communication with the processor 560, including a databus, address bus, and control bus (not shown). The communication bus 555may comprise any standard or non-standard bus architecture such as, forexample, bus architectures compliant with industry standard architecture(“ISA”), extended industry standard architecture (“EISA”), Micro ChannelArchitecture (“MCA”), peripheral component interconnect (“PCI”) localbus, or standards promulgated by the Institute of Electrical andElectronics Engineers (“IEEE”) including IEEE 488 general-purposeinterface bus (“GPIB”), IEEE 696/S-100, and the like.

System 550 preferably includes a main memory 565 and may also include asecondary memory 570. The main memory 565 provides storage ofinstructions and data for programs executing on the processor 560. Themain memory 565 is typically semiconductor-based memory such as dynamicrandom access memory (“DRAM”) and/or static random access memory(“SRAM”). Other semiconductor-based memory types include, for example,synchronous dynamic random access memory (“SDRAM”), Rambus dynamicrandom access memory (“RDRAM”), ferroelectric random access memory(“FRAM”), and the like, including read only memory (“ROM”).

The secondary memory 570 may optionally include an internal memory 575and/or a removable medium 580, for example a floppy disk drive, amagnetic tape drive, a compact disc (“CD”) drive, a digital versatiledisc (“DVD”) drive, etc. The removable medium 580 is read from and/orwritten to in a well-known manner. Removable storage medium 580 may be,for example, a floppy disk, magnetic tape, CD, DVD, SD card, etc.

The removable storage medium 580 is a non-transitory computer readablemedium having stored thereon computer executable code (i.e., software)and/or data. The computer software or data stored on the removablestorage medium 580 is read into the system 550 for execution by theprocessor 560.

In alternative embodiments, secondary memory 570 may include othersimilar means for allowing computer programs or other data orinstructions to be loaded into the system 550. Such means may include,for example, an external storage medium 595 and an interface 570.Examples of external storage medium 595 may include an external harddisk drive or an external optical drive, or and external magneto-opticaldrive.

Other examples of secondary memory 570 may include semiconductor-basedmemory such as programmable read-only memory (“PROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasable read-onlymemory (“EEPROM”), or flash memory (block oriented memory similar toEEPROM). Also included are any other removable storage media 580 andcommunication interface 590, which allow software and data to betransferred from an external medium 595 to the system 550.

System 550 may also include an input/output (“I/O”) interface 585. TheI/O interface 585 facilitates input from and output to external devices.For example the I/O interface 585 may receive input from a keyboard ormouse and may provide output to a display. The I/O interface 585 iscapable of facilitating input from and output to various alternativetypes of human interface and machine interface devices alike.

System 550 may also include a communication interface 590. Thecommunication interface 590 allows software and data to be transferredbetween system 550 and external devices (e.g. printers), networks, orinformation sources. For example, computer software or executable codemay be transferred to system 550 from a network server via communicationinterface 590. Examples of communication interface 590 include a modem,a radio, a network interface card (“NIC”), a wireless data card, acommunications port, a PCMCIA slot and card, an infrared interface, andan IEEE 1394 fire-wire, just to name a few. In one embodiment, thecommunication interface 590 comprises a global positioning satellite(“GPS”) receiver that is configured to receive information from a GPSnetwork and provide that information to the processor or other resourcesof the system 550 for translation into location information on a map.Advantageously, position information on a map may assist in the overallguidance for the vision impaired.

Communication interface 590 preferably implements industry promulgatedprotocol standards, such as Ethernet IEEE 802 standards, Fiber Channel,digital subscriber line (“DSL”), asynchronous digital subscriber line(“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrateddigital services network (“ISDN”), personal communications services(“PCS”), transmission control protocol/Internet protocol (“TCP/IP”),serial line Internet protocol/point to point protocol (“SLIP/PPP”),national marine electronics association (“NMEA”) standards, and so on,but may also implement customized or non-standard interface protocols aswell.

Software and data transferred via communication interface 590 aregenerally in the form of electrical communication signals 605. Thesesignals 605 are preferably provided to communication interface 590 via acommunication channel 600. In one embodiment, the communication channel600 may be a wired or wireless network, or any variety of othercommunication links. Communication channel 600 carries signals 605 andcan be implemented using a variety of wired or wireless communicationmeans including wire or cable, fiber optics, conventional phone line,cellular phone link, wireless data communication link, radio frequency(“RF”) link, or infrared link, just to name a few.

Computer executable code (i.e., computer programs or software) is storedin the main memory 565 and/or the secondary memory 570. Computerprograms can also be received via communication interface 590 and storedin the main memory 565 and/or the secondary memory 570. Such computerprograms, when executed, enable the system 550 to perform the variousfunctions of the present invention as previously described.

In this description, the term “computer readable medium” is used torefer to any non-transitory computer readable storage media used toprovide computer executable code (e.g., software and computer programs)to the system 550. Examples of these media include main memory 565,secondary memory 570 (including internal memory 575, removable medium580, and external storage medium 595), and any peripheral devicecommunicatively coupled with communication interface 590 (including anetwork information server or other network device). Thesenon-transitory computer readable mediums are means for providingexecutable code, programming instructions, and software to the system550.

In an embodiment that is implemented using software, the software may bestored on a computer readable medium and loaded into the system 550 byway of removable medium 580, I/O interface 585, or communicationinterface 590. In such an embodiment, the software is loaded into thesystem 550 in the form of electrical communication signals 605. Thesoftware, when executed by the processor 560, preferably causes theprocessor 560 to perform the inventive features and functions previouslydescribed herein.

The system 550 also includes optional wireless communication componentsthat facilitate wireless communication. Wireless communication can beimplemented over a wireless voice and over a wireless data network. Thewireless communication components comprise an antenna system 610, aradio system 615 and a baseband system 620. In the system 550, radiofrequency (“RF”) signals are transmitted and received over the air bythe antenna system 610 under the management of the radio system 615.

In one embodiment, the antenna system 610 may comprise one or moreantennae and one or more multiplexors (not shown) that perform aswitching function to provide the antenna system 610 with transmit andreceive signal paths. In the receive path, received RF signals can becoupled from a multiplexor to a low noise amplifier (not shown) thatamplifies the received RF signal and sends the amplified signal to theradio system 615.

In alternative embodiments, the radio system 615 may comprise one ormore radios that are configured to communicate over various frequencies.In one embodiment, the radio system 615 may combine a demodulator (notshown) and modulator (not shown) in one integrated circuit (“IC”). Thedemodulator and modulator can also be separate components. In theincoming path, the demodulator strips away the RF carrier signal leavinga baseband receive audio signal, which is sent from the radio system 615to the baseband system 620.

If the received signal contains audio information, then baseband system620 decodes the signal and converts it to an analog signal. Then thesignal is amplified and sent to a speaker. The baseband system 620 alsoreceives analog audio signals from a microphone. These analog audiosignals are converted to digital signals and encoded by the basebandsystem 620. The baseband system 620 also codes the digital signals fortransmission and generates a baseband transmit audio signal that isrouted to the modulator portion of the radio system 615. The modulatormixes the baseband transmit audio signal with an RF carrier signalgenerating an RF transmit signal that is routed to the antenna systemand may pass through a power amplifier (not shown). The power amplifieramplifies the RF transmit signal and routes it to the antenna system 610where the signal is switched to the antenna port for transmission.

The baseband system 620 is also communicatively coupled with theprocessor 560. The central processing unit 560 has access to datastorage areas 565 and 570. The central processing unit 560 is preferablyconfigured to execute instructions (i.e., computer programs or software)that can be stored in the memory 565 or the secondary memory 570.Computer programs can also be received from the baseband processor 610and stored in the data storage area 565 or in secondary memory 570, orexecuted upon receipt. Such computer programs, when executed, enable thesystem 550 to perform the various functions of the present invention aspreviously described. For example, data storage areas 565 may includevarious software modules (not shown) that are executable by processor560.

Various embodiments may also be implemented primarily in hardware using,for example, components such as application specific integrated circuits(“ASICs”), or field programmable gate arrays (“FPGAs”). Implementationof a hardware state machine capable of performing the functionsdescribed herein will also be apparent to those skilled in the relevantart. Various embodiments may also be implemented using a combination ofboth hardware and software.

Furthermore, those of skill in the art will appreciate that the variousillustrative logical blocks, modules, circuits, and method stepsdescribed in connection with the above described figures and theembodiments disclosed herein can often be implemented as electronichardware, computer software, or combinations of both. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, circuits, and steps have beendescribed above generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled persons can implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the invention. In addition, the grouping of functions within amodule, block, circuit or step is for ease of description. Specificfunctions or steps can be moved from one module, block or circuit toanother without departing from the invention.

Moreover, the various illustrative logical blocks, modules, and methodsdescribed in connection with the embodiments disclosed herein can beimplemented or performed with a general purpose processor, a digitalsignal processor (“DSP”), an ASIC, FPGA or other programmable logicdevice, discrete gate or transistor logic, discrete hardware components,or any combination thereof designed to perform the functions describedherein. A general-purpose processor can be a microprocessor, but in thealternative, the processor can be any processor, controller,microcontroller, or state machine. A processor can also be implementedas a combination of computing devices, for example, a combination of aDSP and a microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

Additionally, the steps of a method or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module executed by a processor, or in acombination of the two. A software module can reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of storage mediumincluding a network storage medium. An exemplary storage medium can becoupled to the processor such the processor can read information from,and write information to, the storage medium. In the alternative, thestorage medium can be integral to the processor. The processor and thestorage medium can also reside in an ASIC.

The above description of the disclosed embodiments is provided to enableany person skilled in the art to make or use the invention. Variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the generic principles described herein can beapplied to other embodiments without departing from the spirit or scopeof the invention. Thus, it is to be understood that the description anddrawings presented herein represent a presently preferred embodiment ofthe invention and are therefore representative of the subject matterwhich is broadly contemplated by the present invention. It is furtherunderstood that the scope of the present invention fully encompassesother embodiments that may become obvious to those skilled in the artand that the scope of the present invention is accordingly not limited.

What is claimed is:
 1. A system comprising: a platform comprising ahousing defining an interior space; two or more wheels connected to theplatform and supporting the platform above the ground; a harnessconnected to the platform and positioned to be grasped by an operator;at least two sensors configured to sense information about a localenvironment, wherein each sensor of the at least two sensors isconnected to one of the housing or the harness, and wherein each sensoris configured to sense information associated with a sensor zone of thelocal environment; and a processor configured to analyze informationfrom the at least two sensors and identify one or more objects in aforward path, the processor additionally configured to construct andsend a plurality of messages to the operator, wherein the plurality ofmessages correspond to the identified one or more objects in the forwardpath, and wherein the plurality of messages comprise an audible messagevia a speaker based on information associated with a first sensor zoneof the local environment received from a first sensor of the at leasttwo sensors and a tactile message via the harness based on informationassociated with a second sensor zone of the local environment receivedfrom a second sensor of the at least two sensors.
 2. The system of claim1, wherein the speaker comprises an earpiece.
 3. The system of claim 1,wherein one message of the plurality of messages comprises a secondaudible message sent via a wireless connection to an interface devicebeing worn by the operator.
 4. The system of claim 3, wherein theinterface device being worn by the operator comprises one or more of awatch, a headset, or an earpiece.
 5. The system of claim 1, wherein thetactile message is sent to a portion of the harness being touched by theoperator.
 6. The system of claim 1, further comprising a motorconfigured to rotate the two or more wheels under control of theprocessor.
 7. The system of claim 1, wherein the processor is furtherconfigured to determine a route from a current location to a destinationlocation and provide one or more messages to the operator to guide theoperator from the current location to the destination location.
 8. Thesystem of claim 1, further comprising a tilt preventer configured tobalance the platform above the ground.
 9. A method comprising:receiving, at a device, an input comprising a destination; determining awalking route between a current location and the destination; beginningthe walking route and during the walking route: monitoring the currentlocation to identify a route adjustment; providing a notification of theroute adjustment to a vision impaired person walking the walking route;scanning a current environment of the walking route using at least afirst sensor and a second sensor of a plurality of sensors, the currentenvironment of the walking route comprising a walking surface, andwherein each sensor of the plurality of sensors is configured to monitora sensor zone of the current environment of the walking route; analyzingone or more signals from the plurality of sensors to identify an objectin the walking route; and providing a notification of the object in thewalking route to the vision impaired person walking the walking route,wherein the notification of the object in the walking route comprises anaudible message via a speaker of the device based on one or more signalsassociated with a first sensor zone of the current environment from thefirst sensor and a tactile message via a harness of the device based onone or more signals associated with a second sensor zone of the currentenvironment from the second sensor.
 10. The method of claim 9, whereinthe notification of the object in the walking route comprisesinstructions for avoiding the object.
 11. The method of claim 9, whereinthe notification of the route adjustment comprises one or more of asecond audible message or a second tactile message.
 12. An apparatuscomprising: two or more wheels; an axle extending from at least one sideof the two or more wheels; a harness connected to the axle at a proximalend, the harness extending away from the axle toward a grip positionedat a distal end of the harness, the grip configured to be grasped by anoperator; one or more sensors configured to sense information about alocal environment, wherein each sensor of the one or more sensors isconnected to one of the harness or the grip, and wherein each sensor isconfigured to sense information associated with a sensor zone of thelocal environment; and a processor configured to analyze informationfrom the one or more sensors and identify one or more objects in aforward path, the processor additionally configured to construct andsend a plurality of messages to the operator, wherein the plurality ofmessages correspond to the identified one or more objects in the forwardpath, and wherein the plurality of messages comprise an audible messagevia a speaker based on information associated with a first sensor zoneof the local environment received from a first sensor of the one or moresensors and a tactile message via the harness based on informationassociated with a second sensor zone of the local environment receivedfrom a second sensor of the one or more sensors.
 13. The apparatus ofclaim 12, wherein one message of the plurality of messages comprise asecond audible message sent via a second speaker connected to one of theharness or the grip.
 14. The apparatus of claim 12, wherein one messageof the plurality of messages is a second audible message sent via awireless connection to an interface device being worn by the operator.15. The apparatus of claim 14, wherein the interface device being wornby the operator comprises one of a watch, a headset, or an earpiece. 16.The apparatus of claim 12, wherein the tactile message is sent to thegrip.
 17. The apparatus of claim 12, further comprising a motorconfigured to rotate the two or more wheels under control of theprocessor.
 18. The apparatus of claim 12, wherein the processor isfurther configured to determine a walking route from a current locationto a destination location and provide one or more messages to theoperator to guide the operator from the current location to thedestination location.
 19. The apparatus of claim 12, further comprisinga tilt preventer configured to balance the harness above the ground whenthe grip is not being grasped by the operator.
 20. The system of claim1, wherein each sensor comprises one or more of a Lidar sensor, anultrasonic sensor, an accelerometer, or a proximity sensor.
 21. Theapparatus of claim 12, wherein each sensor comprises one or more of aLidar sensor, an ultrasonic sensor, an accelerometer, or a proximitysensor.